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ABSTRACT 


Billboards are objects, tools or actions, which based on the characteristics 
serve its Own purpose to earn profits, advertise certain people or service, 
and to draw public’s attention by placing it in a very strategic place. It has 
led the government to charge tax on billboards based on its location, 
dimensions, and viewpoints. Therefore, authorized parties have to be able to 
ensure the data authenticity of the proposed billboards. One of the obstacles 
in data verification is the time of billboards measurement process due to its 
size and height from the ground, based on this problem, and we developed a 
system which can measure the dimensions of billboards without physically 
touching it by implementing image processing methods to identify the 
billboards. The implementation is by measuring the dimensions of the 
billboards using perspective concept, then calculates the distance between the 
camera and the object using two-point distance calculation GPS coordinates. 
The results showed that the distance calculation using the GPS method 


generated inaccurate values, whereas the systematic distance method 
generated a result of errors’ range from 0.5 to 25 cm if the image acquisition 
is performed nearly perpendicular to the object. 
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1, INTRODUCTION 

Billboards are objects, tools or actions, which based on the characteristics serve its own purpose to 
earn profits, advertise certain people or service, and to draw public’s attention by placing it in a very strategic 
place [1]. Billboards can be displayed in various types, shapes, and dimensions such as megatrons, 
Videotron, posters, banners, mobile, plaques, etc. 

Nowadays, the role of billboards has become an effective communication system, not only for the 
manufacturers but also for the consumer. The capabilities of billboards and other promotional tools in 
delivering the message to consumers lead them to play essential roles in the success of their company in 
promoting its products and services. 

The growing number of billboards has led the government to charge taxes to billboards based on 
their locations, dimensions, and viewpoints. However, before they are able to measure the billboards, 
they need to obtain the authorization from certain parties to ensure the authenticity of certain billboards. 
Furthermore, due to the large size of it, the government find it hard and dangerous to physically measure 
certain billboards because their tools are not long enough, this is where this system can be implemented as an 
alternative in measuring the billboards. 
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There are many pieces of research on object calculation without measurement instrument using a 
camera or smartphone. However, most of their methods used are either auxiliary or inefficient. One of them 
is the measurement of object distance to camera and object dimension using accelerometer on a smartphone 
by shifting the smartphone according to the length and width of the object to find the object size and to shift 
from the starting point of the smartphone towards the object to calculate the distance [4]. Also, there is 
research where the calculation of distance and object size is calculated using the stereo vision image capture, 
and the distance of an object's size is measured by comparing the width and height of objects obtained from 
two images from two different cameras [5]. 

There is research that uses a more efficient way in which the measurement process uses only one 
camera from a smartphone which then is implemented to calculate the height of a tree using a proportional 
transformation. Where the trees are labeled with two markers to solve the measurement process [6]. 
Other research uses the formula to find the original height of an object with parameters such as the height of 
the object on the image, the height of the sensor, the focal length, the image height, and the distance of the 
object to the camera [7]. 

The taxation fee on billboards is based on its locations, dimensions, and several points of view. 
Authorized parties must ensure the data authenticity of the proposed billboards [11]. One of the obstacles in 
data verification is when the size of the billboards is very big, and the height from the ground 1s considerable. 
So that a system which has the ability to measure the dimensions of the billboards without measuring 
instruments 1s required. 


1.1. Billboard Tax 

The regions’ administration offices issue the taxation of billboards and the imposition of billboard 
tax is not absolute in all regencies or cities in Indonesia. The administration office could choose whether to 
impose the tax to billboards or not. To be collected in a regency or municipality area, the regional 
government must first issue a local regulation concerning billboard tax which shall become the operational 
legal basis in the technical implementation of positioning and collecting billboard tax [2]. The formula to 
calculate the billboard tax is based on the regional regulation of Medan city no. 2 of 2004 [3] as follows. 


Tax = TV + (SV * 25%) 


Where TV (Taxable Value) is the result of multiplication between the area of a billboard, 
the installation period and the rental value of the billboard, SV (Strategic value) represents a value that set at 
the installation location’s point, and 25% is the official billboard tax fee. 


1.2. Perspective Projection 

Perspective projection is a 3-dimensional mapping which is converted into 2 dimensions. 
Moreover, it has the same principle with the way human vision and the camera works. 

From Figure 1, there is a standard formula in searching for a large angle of Q. The formula for the 
large angle of Q can be seen in the equations below. 





Figure 1. Projection of 3D object into a 2D object 
TanQ= = 


TanQ=— 
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Where Y is the original height of the object, Z is the distance between camera and object, y represents the 
object height on camera and f is the focal length on camera. The combination of both equations above creates 
the following equation due to the same size of the angle. 


Me es 
Z 


Rh |< 


Based on the previous equation, the original height of an object can be found by using the former 
equation below and vice versa with the width of an object using the latter equation where W is the original 
width of the object and w is object’s width on images. 





The required variables such as the height of the object on the camera in pixel size, the distance 
between the camera and the object, also the camera's focal length, the original height of the object can be 
found using the latter equation above. 


2. RESEARCH METHOD 

The proposed method for identifying the dimension of a billboard by the image consists of several 
steps which are: Determining the latitude and longitude point based on GPS in the smartphone. Frame per 
frame retrieval from smartphone camera; Gray-scaling; Noise reduction using Gaussian blur [8]. 
Edge detection using Canny edge method[10], then eliminate small blobs with erosion and dilation. 
Contour search and tagging that have the four most substantial angles and widths, then change the position of 
the object perpendicular to the camera with perspective transform. Determine the latitude and longitude and 
calculate the distance between both latitude and longitude point. Search the length and width of the billboard 
by using physics optical formula. After those steps are done based on the obtained length and width of 
billboards can calculate the total billboard tax to be imposed by the government. Detailed explanations of the 
steps undertaken will be explained in the following sections. The general architecture that describes the 
method in this research is shown in Figure 2. 
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Figure 2. General architecture 
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2.1. Data Acquisition 

The data that used in this study are frame by frame image data which collected in real-time mode. 
The image is taken by Android smartphone. Several factors which give impact to the quality of the image are 
camera's quality, environment lightning, the object in front of the billboard, the range between camera and 
billboard, and the angle of the camera to the billboard. For this process, we took 30 billboard image and using 
Caterpillar Handphone with GPS and A-GPS inside, authors also used Garmin GPS to increase our location 
accuracy. While in term of the image we took it using 13 megapixels camera with 1080x1920 pixel 
resolution. 


2.2. Determination of Latitude and Longitude Point 

Latitude and longitude point can be implemented directly using the available code on Android 
programming library which is LocationManager class. At first, LocationUpdate function request will be 
imported to get the information of the smartphone’s location, then calling the getLastkKnowLocation function 
to retrieve the object from Location class where the latitude can be obtained by calling the getLatitude 
function and the longitude point by calling the getLongitude function. 


2.3. Pre-processing 

Before going to the next phase which is the identification of the image, the image has to go through 
several steps in the pre-processing which is gray-scaling, Canny edge detection, image morphology process 
which consists of two sub-process (dilation and erosion). In this pre-process we used the OpenCV library for 
image processing. This particular library is purposed to do image processing in video frames in real- 
time mode. It consists of 400 functions and ranging in 28 areas of image processing and analysis [9]. 


2.3.1 Grayscaling 
Frame (image) that was obtained from the camera will be converted into a grayscale image. 


In OpenC\V, there is a function to convert the RGB image to grayscale image, 1.e., cvtColor (), or it can be 
shown in the formula below [12]: 


G = 0.299(R) + 0.558(G) + 0.114(B) 


The result of the conversion of RGB image value to grayscale image value can be seen in Figure 3. 
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Figure 3. Conversion process of RCB image to a grayscale image 


2.3.2 Canny Edge Detection 
After removing noise on an image, the next phase is edge detection using Canny Edge Detector 
method. The algorithm is as follows: 
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a) Implement noise reduction with 5x5 Gaussian Filter using this equation below : 
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b) Apply Sobel kernel in horizontal and vertical perspective and get its first derivative. By those two 
images we can find the edge gradient and the direction for each resulted pixel as follows: 


m(gradient) = 1G? 5 ae 


Gy 
8 = arctan (=) 
Gy, 


c) Scan all the image pixel to remove unnecessary pixel which is not related to the edge by using non- 
maximum suppression. After that, we will get edges with a thin line. 
d) Do Hysteresis Thresholding, which has a purpose of reconnecting the broken thin edges produced in the 
previous steps. 
In OpenCV, the function to do Canny Edge Detection is Canny(). The explanation of the Canny() 
function is given as follows. 


Canny(src, dst, thres1, thres2, apertureSi1ze) 


The parameters are src for grayscale input image, dst for binary output image with a detected edge, 
thresI represents the initial limit value for hysteresis procedure, thres2 is the final limit values for hysteresis 
procedure and apertureSize as the integer value to determine the matrix dimension on Sobel operator. 
The result of edge detection from a grayscale image using a canny method can be seen in Figure 4. 
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Figure 4. Canny image detection image 


2.3.3 Dilation and Erosion 

The generated edges from canny edge detection are still in dashed and undesirable edge 
line remained. Therefore, dilation and erosion are required to combine all the dashed edges and to eliminate 
small lines that are considered as edges. The functions for dilation and erosion in OpenCV are dilate() and 
erode() respectively. The erode() and dilate() functions have the same parameter yet give different output. 
The erode () and dilate () functions can be described as follows: 


Dilate(src, dst, kernelSize) 
Erode(src, dst, kernelSize) 


The parameters are src as the binary input image, dst for the dilated or eroded output image and 


kernelSize represents the determination of matrix dimension used for dilation or erosion process. The default 
value for kernel size is 3x3. The resulting image of the dilation and erosion process can be seen in Figure 5. 
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Figure 5. Erosion and dilation images 


2.4. Billboard Identification 
This stage 1s the processes where the application can identify the billboard object to be tested. 


2.4.1 Contour Tracing 

In the contour tracing step, edge line that forms a plane (contour) will be stored the. OpenCV 
provides a function to find the contour which is findContour() that produce identified contour points as 
the output. The identified points can be illustrated using drawContour() function. The findContour() and 
drawContour()functions can be described as follows: 


findContour(src, contours, hierarchy, mode, method) 


The parameters are src as the binary input image, contours represents the identified contour 
coordinate point will be stored in a vector, hierarchy is the optional output vector contained information 
about image topology, mode for the contour points detection which is using Imgproc.RETR_LIST for this 
research. While method is the contour approaching method In _ this_ research using 
Imgproc. CHAIN_APPROX_NONE that obtain and store all the contour points. The image of a contour 
detection result can be seen in Figure 6. 
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Figure 6. Identified contour image 


2.4.2 Filter Contour 

After the contour is obtained, a contour that does not match the desired image detection is still 
available. Therefore, it is necessary to process the filter on the contour by searching the most significant area 
and looking for some angles or curves contained in each contour so that the contour obtained is the billboard 
that wants to be measured. Function in OpenCV to calculate the contour area is contourArea() and the one to 
find the angle number is approxPolyDP(). The filter process is done because the billboards must have the 
largest contour area and rectangular shape or have four angles on the image so that the other contour will be 
eliminated. The following explanation of the function described as follows. 


contourArea(contour) 


Parameter : 
Contour : Contour point to which the value of the area is calculated. 


approxPolyDp(contour, approxCurve, epsilon, closed) 
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The parameters are contour as the contour point input, approxCurve for the output in the form of 
coordinates point obtained from the angles, epsilon is the double value that determines the value of accuracy 
in finding angles and closed represents the boolean value that determines whether the angles are connected or 
not. The image of the contour filtering process can be seen in Figure 7. 
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Figure 7. Image of the contour filtering process 


2.4.3 Perspective Settings 

Billboards object are not entirely rectangular because the shots are not always precisely 
perpendicular to the billboards. Therefore, minAreaRect() function is used in OpenCV with billboard contour 
that obtained from contour filtering as the parameter to find a rectangular shape that matches the shape of the 
billboard caught on camera. The rectangularly shaped result of minAreaRect() can be seen in Figure 8. 
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Figure 8. Result of minAreaRect() function 


2.5. Calculation of Billboards Dimension and Tax Charges 
2.5.1 Calculation of Billboard Dimension 

The calculation of billboards dimension can be calculated using perspective projection concept. 
The width and height of the object in the image were obtained from rectangular shaped by minAreaRect() 
function. In this research, the focal length has a default value which is 1740. Suppose that the width of the 
object in the image 1s 652 pixels, the height of 1s 392 pixels and the object distance to the camera is 400 cm, 
then the original width and height of the object can be calculated as follows: 


392 « 400 652 * 400 
—- an Ww =—— 
1740 1740 
__ 156800 __ 260800 
1740 1740 
Y = 90,11 cm W = 149,89 cm 


2.5.2 Calculation of Tax Charges 

Billboard tax charges can be calculated using equation (7). Suppose the sample billboard in Setia 
Budi Rd, which is an elite area, then its rental value Rp 9.000,- and its strategic value at Rp 30.000.000,-. 
Therefore, the tax charges that need to be paid can be calculated as follows. 


149,89 « 90,11 
Tax = ( (aa | * 365 * 9.000 + (30.000.000 * 0.25 ) 


Tax = 4.436.914,12515 + 7.500.000 
Tax = Rp 11.936.914,12515 anually 
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As the tax charge calculation finish, then the data will be inputted into the system, which will be 
used to be compared in another inspection. This is to solve the problem that sometimes the size of the 
billboard and the tax calculation which was filled inside the tax paper is written differently from its actual 
size in the field. 


3. RESULTS AND ANALYSIS 

In this section, we described the result of the system based on the error or difference between its 
original size and predicted size by the system. We also calculate the accuracy of the system, using 30 
different billboards of different size and location. 

Before we proceed, Figure 9 and Figure 10 shows the final interface before the data send to 
the server. Figure 9 shows the final capture of the billboard, wherein this stage we already have the billboard 
measurement. 
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In Figure 10 we took two coordinates based on two different location, they are a billboard location, 
and the photo was taken location. These two location is used to measure distance so that we can get the real 
size of the billboard, if the GPS is not working correctly we can use the column for manual distance based on 
estimation. Once we get the generated data, it will be saved and send to the server. 

Based on the results in Table 1, we can see that the between billboard and image acquisition plays 
an important part. The distance usually in 5-20 meters between those two points. The distance will contribute 
to the angle of the image. We have to consider to take billboard image as flat as possible. 

We can see that based on the table there 1s a different error in terms of width and height of the image 
if the billboard’s size larger than 300 cm, billboard no 4, 12, 21, 22, has error value of + 10 - 20 cm, which 
will affect tax calculation from the system. However, in the real-life application, the tax calculation and size 
measurement of the billboard from our system is considered estimation and tools for a legal decision from the 
right authority. 

If we assume the lousy result that has error distance value is larger than 15 cm, we can say that out 
of 30 testing images; we will have 25 good results and five bad result, which means we get 83,3% accuracy 
value. However, it is still acceptable in the real-life implementation, because usually, the tax authority will 
classify the size of the billboard into a particular category. 

The GPS accuracy plays an essential role to measure the correct size measurement. 
The dependencies of an accurate device is a must. GPS in a standard smartphone sometimes will give lousy 
result also. However, it usually gives a problem with the first data taken, while once recorded in the system it 
can be considered for the next data taken. Aside from that, image processing in the system gives good 
accuracy. However, improvement is a must. Some of the billboard images have noise because of other 
objects in front of the billboard, such as cable, leaves, tree, or another billboard object. This problem will 
become our next research, which we assume can be solved if we add machine learning to the system. 
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Table 1. Pre-processing Result 


Width (cm) Height (cm) Predicted Width(cm) Predicted Height(cm) Width Error (cm) Height Error (cm) 
240 50 247.6552 51.4942 -7.6552 -1.4942 
240 120 240.5517 118.9885 -0.5517 +1,0115 
140 50 137.6034 48.6437 +2,966 +1,3563 
A400 100 410.2989 104.2471 -10,2989 -4,2471 
120 80 122.3621 74.0115 -2,3621 +5,9885 
A400 130 402.0402 128.9655 -2,0402 +1,0345 
150 150 152.3403 151.0938 +2.3403 +1.0938 
200 150 205.0933 147.9093 +5.0933 -2.0907 
150 90 157.0879 92.9312 +7.0879 +2.9312 
300 200 303.0393 198.0382 +3.0393 -1.9618 
350 A400 359.0545 415.8374 +9.054 +15.8374 
450 200 ATT 1232 205.9384 +27.7232 +5.9384 
200 200 203.6734 194.2311 +3.6734 -5,7689 
240 220 240.9853 203.2182 +0.9853 -16,7818 
125 90 126.0821 97.4932 +1.0821 +7.4932 
140 100 146.3766 100.2345 +6.3766 +0.2345 
200 150 O35 35. 153.3236 -4,2643 +3.3236 
200 150 196.9374 151.3947 -3,0626 +1.3947 
300 150 291.8115 148.0517 -8,1885 -1,9483 
A400 250 400.6584 290.3238 +0.6584 +5.3238 
450 250 472.8374 253.4933 +22.8374 +3.4933 
A400 250 423.2329 292.3233 +23.2329 +2,3233 
300 200 307.8943 212.6832 +7.8943 +12.6832 
200 150 205.0023 153.3948 +5.0023 +3.3948 
150 90 152.9384 93.4834 +2.9384 +3.4834 
300 150 301.9383 150.2649 +1.9383 +0.2649 
400 100 415.6254 100.3495 +15.6254 +0.3495 
240 100 248.3845 107.0845 +8.3845 +7.0845 
250 150 250.4383 155.3948 +0.4383 +5.3948 
140 50 147.5323 52.2356 +7 5323 +2.2356 

CONCLUSION 

Following the research and its experimental activities conducted, it 1s concluded that: 

GPS plays an essential part as a reference in finding distances due to its inefficiency and inaccuracy in 


obtaining the distance point. 


b) The calculation of the billboard’s size can be done using perspective projection concepts, as long as the 
image taken by the user is perpendicular to the billboard. 

c) The recognition of billboards using image processing is strongly opposed due to many factors such as 
lighting, the surrounding environment, a contrast of billboards color with its surrounding environment, 
and the placement of billboards that make the edge detection unsuitable as desired, resulting in 
unrecognized billboards. 

Based on the research conducted, the suggestions that expected to do in further research as follows: 

a) Use accurate GPS device so the measurement will perform well. 

b) In image processing we have several issues in the future, they measure the billboard regardless of the 
angle of the captured image, eliminates other noise, implementing other detailed segmentation to get 
different and better result. 

c) Implementing machine learning to recognize the billboard image. 
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